The present invention relates to head mounted video displays for presenting virtual environments, and more particularly to a binocular head-mounted video display with full field of view and high resolution video images.
Traditionally, displays of virtual environments have been used for entertainment purposes, such as presenting the environments for the playing of various video games. More recently, such displays have been considered for other applications, such as possible tools in the process of designing, developing, and evaluating various structures and products before they are actually built. The advantages of using virtual displays as design and development tools include flexibility in modifying designs before they are actually built and savings in the costs of actually building designs before they are finalized.
To be a useful and valid design and development tool, however, a virtual display system must be capable of generating high fidelity, interactive, virtual environments that provide correct xe2x80x9cfeelings of spacexe2x80x9d (FOS) and xe2x80x9cfeelings of massxe2x80x9d (FOM). Such a system must also allow users to function xe2x80x9cnaturallyxe2x80x9d within the virtual environment and not experience physical or emotional discomfort. It must also be capable of displaying a virtual environment with dynamics matched to the dynamics of human vision and motor behavior so there is no perceptible lag or loss of fidelity.
FOS and FOM are personal perceptual experiences that are highly individual. No two people are likely to agree on FOS and FOM for every environment. Also, there are likely to be variations between people in their judgments of FOS and FOM within a virtual environment, as compared to FOS and FOM in the duplicated real environment. Thus, preferably a virtual display system will provide feelings of space and mass that are based on a more objective method of measuring FOS and FOM that does not rely on personal judgments of a particular user or a group of users.
With regard to human vision, typically there are xe2x80x9cnatural behaviorsxe2x80x9d in head and eye movements related to viewing and searching a given environment. One would expect, and a few studies confirm, that visual field restrictions (e.g., with head mounted telescopes) result in a limited range of eye movements and increased head movements to scan a visual environment. Forcing a user of a virtual display system used as a design and development tool to adapt his or her behavior when working in a particular virtual environment could lead to distortions of visual perception and misjudgment on important design decisions. Thus, the ideal virtual display system will have sufficient field-of-view to allow normal and unrestricted head and eye movements.
Simulator sickness is a serious problem that has limited the acceptance of virtual reality systems. In its broadest sense, simulator sickness not only refers to feelings of dizziness and nausea, but also to feelings of disorientation, detachment from reality, eye strain, and perceptual distortion. Many of these feelings persist for several hours after use of a system has been discontinued. Most of the symptoms of simulator sickness can be attributed to optical distortions or unusual oculomotor demands placed on the user, and to perceptual lag between head and body movements and compensating movements of the virtual environment. Thus, preferably a virtual display system will eliminate simulator sickness.
One technology commonly used to present virtual environments are head mounted video displays. A head mounted display (xe2x80x9cHMDxe2x80x9d) is a small video display mounted on a viewer""s head that is viewed through a magnifier. The magnifier can be as simple as a single convex lens, or as complicated as an off-axis reflecting telescope. Most HMDs have one video display per eye that is magnified by the display optics to fill a desired portion of the visual field.
Since the first HMD developed by Ivan Sutherland at Harvard University in 1968, there has always been a trade-off between resolution and field of view. To increase field of view, it is necessary to increase the magnification of the display. However, because video displays have a fixed number of pixels, magnification of the display to increase field of view is done at the expense of visual resolution (i.e., visual angle of the display pixels). This is because magnification of the display also increases magnification of individual display pixels, which results in a trade-off between angular resolution and field of view for HMDs that use single displays. Normal visual acuity is 1 minute of arc (20/20). Legal blindness is a visual acuity of 10 minutes of arc (20/200). The horizontal extent of the normal visual field is 140xc2x0 for each eye (90xc2x0 temporal and 50xc2x0 nasal). Thus, to fill the entire visual field with a standard SVGA image, one must settle for visual resolution that is worse than legal blindness.
One attempt to develop an HMD with both high visual resolution and a large monocular field of view was made by Kaiser Electro-Optic, Inc. (xe2x80x9cKEOxe2x80x9d) under a contract with the Defense Advanced Research Projects Agency (xe2x80x9cDARPAxe2x80x9d). KEO developed an HMD that employed a multi-panel xe2x80x9cvideo wallxe2x80x9d design to achieve both high resolution with relatively low display magnification and wide field of view. The HMD developed by KEO, called the Full Immersion Head Mounted Display (xe2x80x9cFIHMDxe2x80x9d), had six displays per eye. Each display of the multiple displays forming the video wall was imaged by a separate lens that formed a 3xc3x972 array in front of each eye. The horizontal binocular field of view of the FIHMD was 156xc2x0 and the vertical was 50xc2x0. Angular resolution depended on the number of pixels per display. The FIHMD had 4 minarc per pixel resolution.
FIG. 1 is a plan view of the FIHMD, while FIG. 2 shows the optics 10 of the FIHMD. These optics included a continuous meniscus lens 12 (xe2x80x9cmonolensxe2x80x9d) between the eye (not shown) and the six displays 14 and a cholesteric liquid crystal (xe2x80x9cCLCxe2x80x9d) filter 16 for each display 14. The meniscus lens 12 served as both a positive refracting lens and as a positive curved mirror. The CLC 16 reflected light from the displays 14 that passed through the meniscus lens 12 back onto the lens 12 and then selectively transmitted the light that was reflected from the lens"" curved surface. Some versions of the FIHMD optical design employed Fresnel lenses as part of the CLC panel to increase optical power. This so-called xe2x80x9cpancake windowxe2x80x9d (also called xe2x80x9cvisual immersion modulexe2x80x9d 18 or xe2x80x9cVIMxe2x80x9d), shown in FIG. 3, provided a large field of view that was achieved with reflective optics while folding the optical paths into a very thin package.
The FIHMD 20, shown in FIG. 1, could not provide a satisfactory full field of view. The FIHMD had limitations imposed by its use of the VIM optics and the requirement for adequate eye relief to accommodate spectacles 22. The radius of curvature of the meniscus lens 12 dictated the dimensions of the VIM 18 and, coupled with the eye relief requirement, determined the location of the center of curvature of display object space. Although no documentation is available that discusses the rationale for the design, as illustrated in FIG. 1, it appears that the centers of VIM field curvature 24 for the FIHMD 20 were set in the plane of a user""s corneas. If the centers of the two VIM fields are separated by the typical interpupillary distance (68 mm), then the centers are located 12 mm behind the lens 23 of spectacles 22. This is the usual distance from a spectacle lens to the surface of the cornea. Because of this choice of centers, the FIHMD 20 had problems with visibility of seams between the displays 14 and with display alignment.
Normally, when the visual angle subtended by an object is measured, the apex of the cornea is used as the reference. Technically, for paraxial rays, the anterior nodal point of the eye, which is 7.2 mm posterior to the cornea, should be used as the reference. However, most object distances are large enough that a 7.2 mm error in the choice of reference is negligible. The FIHMD used the apex of the cornea as the center of curvature for its display field. But, in the case of FIHMD, the distances are small and the 7.2 mm error is significant.
Using the apex of the cornea as a reference overestimates the size of the visual field in an HMD. Such an overestimation only affects visual field and visual resolution specifications. However, a more serious problem is encountered when a viewer moves his eyes. Because the eye rotates about a point 13.5 mm posterior to the cornea, the apex of the cornea is translated as the eye rotates and the geometry of the optical system changes.
Display image overlap and image alignment in the FIHMD had to change with eye movements because the center of rotation of the eye is located 13.5 mm posterior to the front surface of the cornea. FIG. 4 is a plan view of a schematic ray tracing model 26 of the FIHMD optical configuration. In the model 26, the VIM modules 18 are replaced with single Fresnel lenses 28 that image the flat panel displays 14 at infinity. For on-axis viewing of the nasal-most display 14A (top of FIG. 4), there is sufficient image overlap to produce a seamless montage 30 in the retinal image 32. Consequently, if a user of the FIHMD stares straight ahead, he will experience a wide field of view and a seamless image.
When a user rotates his eye 34 to change the direction of gaze, the display images shift on the retina 35 by different amounts and the seamless montage breaks up. FIG. 5 illustrates, with the same model used in FIG. 4, the effects of eye movements in the FIHMD on the retinal image 32. In this example, the observer has turned his eye 34 temporally to fixate the center of the display 14B located 30 degrees away from the display 14A that was fixated in FIG. 4. Because the eye 34 rotates about a point 13.5 mm behind the cornea, instead of at the corneal apex 36 (which is the center of the sphere tangent to the optical and display array), the eye 34 translates relative to the optic axes of the FIHMD. To fixate the center of the 30xc2x0 display, the eye rotates 23xc2x0 and, relative to the optic axes, translates 1 mm on the z-axis and 5.2 mm on the x-axis. This translation of the eye""s optics relative to the display optics introduces prism and vignetting. The result is that the display images on the retina were separated, shown as 30A in FIG. 5, by large gaps 38 ( greater than 5xc2x0), which are represented by the black areas in the ray tracing model 26. As the observer looks around, he would see gaps between the displays changing in prominence, and he most likely would see movement of the display images (particularly with smooth pursuit eye movements). Thus, the FIHMD suffered from montage image break-up as a user rotated his eyes within the device.
It is an object of the present invention to provide a head mounted display for presenting virtual environments with improved high resolution and full field of view.
It is another object of the present invention to provide a head mounted display for presenting virtual environments with improved high resolution and full field of view and that satisfies the requirements of human vision and motor behavior.
It is a further object of the present invention to provide a head mounted display for rendering virtual environments with high enough fidelity to produce correct feelings of space and mass and without simulator sickness.
It is yet another object of the present invention to provide a head mounted virtual environment video display which solves the montage image break-up problem of the FIHMD.
The present invention solves the montage image break-up problem of the FIHMD, while achieving both high visual resolution and a full field of view. The present invention uses an optical system in which the video displays and corresponding lenses are positioned tangent to hemispheres with centers located at the centers of rotation of a user""s eyes. Centering the optical system on the center of rotation of the eye is the principal feature of the present invention that allows it to achieve both high fidelity visual resolution and a full field of view without compromising visual resolution.
The HMD of the present invention uses a simpler optical design than that used by the FIHMD. The present invention uses an array of lens facets that are positioned tangent to the surface of a sphere. The center of the sphere is located at an approximation of the xe2x80x9ccenter of rotationxe2x80x9d of a user""s eye. Although there is no true center of eye rotation, one can be approximated. Vertical eye movements rotate about a point approximately 12 mm posterior to the cornea and horizontal eye movements rotate about a point approximately 15 mm posterior to the cornea. Thus, the average center of rotation is 13.5 mm posterior to the cornea.
The present invention also uses a multi-panel video wall design for the HMD""s video display. Each lens facet images a miniature flat panel display which can be positioned at optical infinity or can be adjustably positioned relative to the lens facet. The flat panel displays are centered on the optical axes of the lens facets. They are also tangent to a second larger radius sphere with its center also located at the center of rotation of the eye. The present invention also includes high resolution and accuracy head trackers and built-in eye trackers. One or more computers having a parallel graphics architecture drives the head mounted display and uses data from these trackers to generate high detail 3D models at high frame rates with minimal perceptible lag. This architecture also optimizes resolution for central vision with a roaming high level of detail window and eliminates slip artifacts associated with rapid head movements using freeze-frame. The result is a head mounted display that renders virtual environments with high enough fidelity to produce correct feelings of space and mass, and which does not induce simulator sickness.